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Abstract 

University and high school students not pursuing a science, technology, engineering, and/or mathematics (STEM) 
course of study demonstrate less developed scientific reasoning than their STEM-based peers. Previous studies show 
that the majority of non-STEM students can be classified as either concrete operational or transitional reasoners in 
Piaget’s theory of cognitive development, whereas in the science student population formal operational reasoners are 
far more prevalent. In this paper, we will look at the literature concerning the non-STEM population of students and 
their development in scientific reasoning. We will also discuss the development and implementation of activities 
designed to target hypothetico-deductive reasoning, coordination of theory and evidence, and thinking about the 
relative value of evidence. We will also discuss an authentic research experience we have begun introducing in our 
classrooms to facilitate the transition from transitional to formal operational reasoning. 
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Introduction 

University and high school students not pursuing a science, technology, engineering, and/or 
mathematics (STEM) course of study demonstrate less developed scientific reasoning than their 
STEM-based peers (Moore, 2012). Significant work has gone into developing research-verified 
pedagogical methods for pre-service teachers and the algebra- and calculus-based physics courses 
typically populated by natural and physical science majors; (McDermott, 1999) however, there is 
significantly less volume in the literature concerning the non-science, general education 
population (Etkina, 2004). This is quickly changing, and large, repeatable gains on concept tests 
are being reported, specifically within the astronomy education community (Bailey, 2003). 
However, we may be losing sight of what is arguably the most important goal of such a course: 
development of scientific reasoning. Are we teaching this population of students to think l ik e 
scientists? 

A recent discussion in the US journal The Physics Teacher has led us to examine the 
central focus of our courses for non-STEM majors (Sobel, 2009; Lasry, 2009). Like Lasry, 
Finkelstein and Mazur, we certainly do not believe that this population of students are “too 
dumb” for physics, or that physics is in a “different category” of hard accessible only to certain 
students such as science majors. However, there are very real differences in the two populations, 
especially when considering interest level, formal preparation, and prior development of 
scientific reasoning skills. It is reasoning that we focus on in this project, since we believe 
development of scientific reasoning should be a central goal for these types of courses. 

In particular, reasoning and metacognition development are essential if we hope to elevate 
students to “expert-like” status with respect to problem solving, understanding and applying 
abstract concepts, and shifting between multiple representations (Etkina, 2004; Kohl, 2008; 
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White, 1998). However, non-science majors enter the classroom with a disadvantage not 
necessarily shared by their self-selecting science major peers. Non-scientists struggle with basic 
scientific reasoning patterns, which can hinder their growth in the course (Moore, 2012; Hogan, 
2001; Reif, 1991). Acknowledgement of this dramatic difference in reasoning ability is important 
for development of good pedagogy, considering scientific reasoning has been linked to student 
gains in conceptual knowledge for both non-scientist and scientist populations (Moore, 2012; 
Coletta, 2005). Even with significant disadvantages, gains in content knowledge can still be 
obtained in conceptual physics and astronomy courses, especially when those courses are 
designed around a research-verified, active-engagement curriculum. However, gains in content 
knowledge do not necessarily lead to gains in scientific reasoning (Moore, 2012; Pearsall, 1997). 
In fact, the content-specific education literature in other disciplines suggests that explicit 
intervention is necessary to improve reasoning (Etkina, 2004; White, 1998; Lawson, 2000; Blank, 
2000). It is this explicit intervention that is currently lacking in many pedagogical models that 
address this student population in physics and astronomy, specifically those models practical for 
implementation with a large student-to-faculty ratio (Murthy & Etkina, 2004). 

Development of scientific reasoning is not only a necessary means to an end (making their 
thinking more scientific so that they can better grasp the content); it is also a justifiable end in 
and of itself. We should expect our courses to affect our students beyond the classroom. 
Particularly for non-scientists, a broader approach should be expected since these types of 
courses are typically their terminal experience in formal science education. Scientific reasoning 
and metacognitive development are often required for effective decision-making and problem 
solving far outside the typical scientific context (Reif, 1991; Greenhoot, 2004; Overton, 1990). 

In this paper, we first evaluate the literature with respect to postsecondary scientific 
reasoning abilities of non-STEM students. We then discuss the development, implementation, 
and efficacy of explicit reasoning and metacognition interventions designed to improve students’ 
approaches to and thinking about science. 

Scientific Reasoning, Concept Construction, and Cognition 

What exactly constitutes scientific reasoning is both complex and debatable. Lawson suggests 
that scientific reasoning has a structure that is chiefly hypothetico-deductive in nature and 
consisting of interrelated aspects, such as proportional reasoning, control of variables, probability 
reasoning and correlation reasoning (Lawson, 1982, 2005). Inductive and deductive process are 
involved, with some researchers intimately linking reasoning with the process of drawing 
inferences from initial premises (Overton, 1990; Holyoak, 2005). More recently, Kuhn has 
suggested that scientific reasoning is more than inductive inference, but a truth-seeking social 
process that involves the coordination of theory and evidence (Kuhn, 2004). Kuhn and others 
specifically suggest that reasoning process cannot be separated from prior knowledge (Kuhn, 
2004; Schaubel, 1996; Dunbar, 2000; Lawson, Clark, et al., 2000). The learning of content and 
reasoning development have been linked in the physics education literature, which is consistent 
with these studies (Moore, 2012; Coletta, 2005). Specifically, content gains are significantly 
more difficult to achieve with underprepared students vs. well-prepared students, and gains in 
reasoning only materialize with explicit intervention (Moore, 2012). 
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The epistemological beliefs of a student can also dramatically influence scientific 
reasoning. Specifically, Edmondson and Novack have demonstrated that students rely on more 
rote learning strategies when they hold a positivist view of science, where they regard science as 
an existing body of knowledge to be discovered by authority and/or passive observation 
(Edmondson & Novack, 1993). Students holding a constructivist epistemology typically are far 
better at making connections and constructing experiments; they also demonstrate greater ability 
to achieve significant gains in conceptual knowledge (Moore, 2012; Edmondson & Novack, 
1993; Tsai, 1998). Physics-specific studies of students’ personal epistemologies “in the moment” 
show similar observations (Roth, 1997; Zeineddin, 2010), where significant focus has been on 
metacognition/epistemological framing with respect to problem solving (Bing, 2008, 2009; 
Tuminaro, 2007). Specifically, Bing and Redish argue that demonstrating the ability to switch 
between epistemological resources in approaching a problem is an indicator of what they call the 
“journeyman-expert transition” (Bing & Redish, 2012). As we will discuss, we have made similar 
observations with both our freshmen physics majors and non-STEM students in our most recent 
courses; however, our observations are made in the context of experiment construction and 
application of the hypothetico-deductive model. In particular, student views about science greatly 
influence their ability to “create new knowledge,” where students limit themselves to certain 
modes of knowledge construction. Even students that hold more constructivist views of science 
overall can limit themselves in a similar way when confronted with specific problems and 
situations. 

Piaget’s theory of cognitive development includes classification into two formal reasoning 
levels (concrete operational and formal operational) with a transitional stage between the two 
(Inhelder & Piaget, 1958). Students classified as mostly concrete operational are characterized by 
their appropriate use of logic; however, they struggle with solving problems outside of a concrete 
context, demonstrating significant difficulty with abstract concepts and hypothetical tasks. 
Transitional reasoners fall between the other two classifications where they find success with 
hypothetical tasks in some contexts. Formal operational reasoners begin to think abstractly, 
reason logically, and draw conclusions from available information. Furthermore, unlike the 
concrete operational and transitional reasoner, they are able to apply appropriate logic to 
hypothetical situations in most contexts. In this way, formal operational reasoners can begin to 
think like a scientist, and specifically develop strong hypothetico-deductive reasoning. As will be 
discusses, the students observed in our studies are predominantly transitional or concrete 
operational reasoners, which creates a more difficult environment in which to make learning 
gains. 

Typically, formal schooling in science does not resemble the actual practice of science by 
scientists (Reif & Larkin, 1991). For development of new activities, we have worked within the 
framework of cognitive apprenticeship, where the actual process of science is the means by 
which the interventions are structured (Brown, Collins, & Duguid, 1989; Collins, et al., 1987). 
Barab and Hay describe cognitive apprenticeship loosely as “doing science at the elbows of 
experts,” and describe cognitive apprenticeship as having the following features (Barab & Hay, 
2001 ): 

1. Learners do science to address science dilemmas. 

2. Scientific knowledge/practice is situational constructed and socially negotiated. 

3. Learning is participatory, including teachers, scientists, and peers. 
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4. Practice and outcomes are authentic to and owned by the learner and the 
community, and are in response to real-world problems/needs. 

5. Participants become a part of a community of practice. 

6. Formal opportunity and support for both reflection-in-action and reflection-on- 
action. 

Formal Reasoning Levels and Patterns for Non-STEM Students 

Using the Lawson’s Classroom Test of Scientific Reasoning (LCTSR; Lawson, 1978), we have 
investigated our students’ weaknesses with respect to scientific reasoning. Specifically, we have 
looked at the formal reasoning level for the average non-STEM student enrolled in either our 
conceptual physics or astronomy courses, and we have determined which scientific reasoning 
patterns present the most difficulty for this population. Our specific methodology is discussed in 
greater detail elsewhere (Moore & Rubbo, 2012). Here, we present a brief overview of our 
previous studies on formal reasoning level to provide the context in which the explicit 
interventions have been developed that are the central focus of this paper. 



LCTSR 

N 

St. 


% 


dev. 

STEM 

74 

1208 

18 

Non-STEM 

54 

109 

17 


Table I: Average LCTSR scores for STEM and non-STEM students 
(Ran. 2009: Moore A Rubbo. 2012). 


Students in our conceptual physics and astronomy courses score significantly lower on the 
LCTSR compared to students enrolled in courses typically populated with science majors. The 
LCTSR assesses reasoning patterns such as proportional reasoning, control of variables, 
probability reasoning, correlation reasoning and hypothetico-deductive reasoning. Table I shows 
average LCTSR pre-instruction scores (N = 1208, avg = 74.2%) for freshman science and 
engineering majors enrolled in a calculus-based introductory physics course, as reported by Bao, 
et al. (Bao, 2009). The LCTSR was also administered to students taking a conceptual physics or 
astronomy course at two regional, comprehensive universities in the United States during the past 
four years (Moore & Rubbo, 2012). As shown in Table I, this student population scores 
significantly lower (N = 109, avg. = 54%). 

Using individual student scores on the LCTSR, we classified students into three Piagetian 
formal reasoning categories: concrete operational (CO), transitional (T), and formal operational 
(LO), as shown in figure 1 (Inhelder & Piaget, 1958). As described by Lawson, students scoring 
below 25% on the LCTSR were classified as concrete operational reasoners, students scoring 
between 25% and 58% were classified as transitional reasoners, and students scoring above 58% 
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were classified as formal operational reasoners (Lawson, Clark, et al., 2000). A significant 
majority of non-STEM students (56%) are classified as transitional reasoners. This observation is 
consistent with previous studies of the general education population in introductory biology 
courses for the non-major (Johnson & Lawson, 1998). 

Population averages for specific scientific reasoning patterns as assessed by the LCTSR are 
discussed in previous work (Moore & Rubbo, 2012). Specifically, students within the observed 
population demonstrated significant difficulty with proportional reasoning, isolation of variables, 
and hypothetico-deductive reasoning. Scores on LCTSR questions designed to test application of 
hypothetico-deductive reasoning average to about 30%. Proportional reasoning was the reasoning 
pattern on which students scored the lowest (25%), which could be attributable to poor 



Fig. 1: Distribution of formal reasoning level. Concrete operational 
(CO), transitional (T), and formal operational (FO) reasoning levels 
are shown (Moore, 2012). 


preparation in mathematics. Most students demonstrate some proficiency with correctional and 
probabilistic reasoning. In this paper, we focus on interventions designed to develop hypothetico- 
deductive reasoning within the framework of cognitive apprenticeship. We will also discuss the 
transition from transitional to formal operational reasoning, and students’ evaluation of the 
relative value of evidence and the coordination of data with models. 

Explicit Intervention for Reasoning and Metacognition 

Through years of observations in courses where we have adopted research-verified pedagogy, we 
have show great success with increasing learning gains on content-based assessments. However, 
we have only been recently successful with transitioning students from transitional to formal 
operational reasoners via explicit reasoning and metacognitive interventions. In this section, we 
will detail some of the activities that we have developed, how they can be implemented, and how 
the instructor can recognize a shift in student approach during their implementation. 
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If... and ... then: Making hypothetico-deductive reasoning explicit 

To improve scientific reasoning ability, we have recently begun to incorporate explicit 
instruction in reasoning patterns, with focus on those where deficiencies have been found, such as 
hypothetico-deductive, proportional, and isolation of variables (Moore & Rubbo, 2012). Here, we 
will focus on hypothetico-deductive reasoning, and making this type of thinking formal within 
the conceptual physics course. Lawson presents a series of activities that lead students through 
the process of constructing good “if ... and ... then” (IAT) statements in various fields of 
knowledge (Lawson, 2000). In this way, hypothetico-deductive reasoning is being made explicit. 
We have begun introducing these types of activities within our course with preliminary success. 

In our curriculum students design an experiment that would address the question “does a 
light bulb use up current?” We have begun to formalize the process by forcing students to 
construct an appropriate IAT statement that is specifically designed to potentially falsify or 
support a claim. Before we ask students to prepare their own statements for this particular 
situation, we instruct them on the proper syntax for IAT construction. Specifically: 

If... hypothesis/model, 
and ... testing experiment, 

then ...result assuming hypothesis/model to be true. 

With respect to the question “does a light bulb use up current?,” we would like to see students 
respond similar to the following: 

If... current is used up in a bulb, 
and ... we measure current on both sides, 
then .. .the current will be less on one side. 

Based on observations from a test class consisting of 15 students, we have found that students 
initially struggle with IAT statement construction but gradually improve. The following are 
examples of student responses to the light bulb task at the beginning of the course: 

SI: If we add another bulb to the circuit, and we observe the brightness to decrease, 

then current is used up in bulbs. 

S2: //'Ohm's Law is valid, and the current decreases when the voltage decreases, then 

current is used up in bulbs. 

S3: If we conduct a controlled experiment varying only the current, and the bulbs 

behavior does not depend on the current, then current is not used up in bulbs. 

We can see from these responses two patterns: (1) students struggle to properly order the 
hypothesis, testing experiment, and result even after explicit (although passive) instruction, and 
(2) students struggle to identify an appropriate testing experiment for the given hypothesis/model. 

Students’ struggles with ordering may result from conflating the hypothesis/model with the 
result. Furthermore, we observe significant difficulties among the non-science population with 
devising appropriate testing experiments for a given hypothesis independently. Typically, 
students take one of the following three approaches: 
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1. The student devises a testing experiment that does not test the hypothesis/model 
under review. The actual experiment may be valid in a different context, but does 
not test within the immediate context. This approach is exhibited by SI and 
partially by S3. 

2. The student appeals to the authority of a specific rule/law/model, person, or 
previous passive observation. This approach is exhibited by S2. 

3. The student recites previously learned jargon concerning the “scientific method,” 
such as independent and dependent variables, controlled experiments, etc. without 
explanation or definition within the immediate context. This approach is partially 
exhibited by S3. 

We attempt to provide an environment within the classroom where students can construct 
their own process for developing hypotheses/models and testing experiments that probe these 
hypotheses/models. Specifically, we utilize scaffolded activities beginning with the passive 
instruction on proper IAT syntax, advancing to guided activities, and ending with independent 
practice. 

In a different test class than that discussed above (again, approximately 15 students) we had 
students evaluate the three approaches listed above in a guided activity, and discover for 
themselves the limitations in these approaches. We asked the students to discuss in groups the 
following IAT statement: 

“If current is used up in bulbs, and we conduct a controlled experiment varying only the 
independent variable, then the behavior of the bulb will not depend on the independent 
variable.” 

Specifically, we asked them to come up with a way of performing the proposed testing 
experiment with the equipment provided. Not surprisingly, the class comes up with as many 
testing experiments as there are student groups. This allowed for a whole-class discussion about 
the need for a definition of “independent variable” and “controlled experiment.” Several students 
commented concerning the link between the hypothesis/model and the result of the testing 
experiment. We then had students discuss the following IAT statement: 

“If current is used up in bulbs, and we add another bulb to the circuits, then the bulb will 
get dimmer.” 

Within the activity, we instructed each group to conduct the testing experiment in one of 
the following two ways: (1) by adding the extra bulb in series, and (2) by adding the extra bulb in 
parallel. After performing the experiments, we had two groups report on their results, with one 
group having followed the series protocol, and the other group having followed the parallel 
protocol. Neither group knew that the other group was following a different protocol. The series 
group determined that the testing experiment supported the hypothesis, and the parallel group 
reported that the testing experiment falsified the hypothesis. This set up a broader discussion of 
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the terms “support” and “falsify,” specifically, the concept that supporting evidence does not 
constitute proof. 

Hypothesis/Model Testing Experiments and Metacognitive Challenges 

Explicit discussions on reasoning in a guided-inquiry style classroom must be approached with 
care, since it is easy for struggling students to appear to be making connections, when in fact they 
are merely parroting by rote the reasoning of other individuals in their group. This became 
apparent to us when we discovered students doing poorly on individual reasoning tasks post¬ 
instruction, when during participation in class they seemed to correctly approach the activities 
requiring these skills (Moore & Rubbo, 2012; Moore, 2009, 2010, 2011). 

To investigate this discrepancy, we asked students the following open-ended question about 
a basic electric circuit before instruction during a smaller enrollment instance of one of our 
courses: 


“A single bulb circuit consists of a battery, two wires and a bulb arranged such that the 
bulb is lit. A student wonders whether it matters which part of the bulb is connected to the 
‘+’ sign on the battery. Describe an experiment that the student can do to make this 
determination.” 

Follow-up: “What result of your experiment would tell you that the orientation of the 
battery does matter?” 

The following are typical student responses to the first question: 

SI: The orientation of the battery does not matter, so rebuild the circuit in exactly the 

same way to show that the orientation does not matter. 

S2: Get a different battery, bulb and set of wires and build a second circuit with the 

battery in the opposite orientation. 

S3: Using the same components, turn the battery in the opposite orientation. 

S4: Conduct a controlled experiment, varying only the number of batteries to see if 

that makes a difference. 

S5: Redo the experiment, but make sure it is controlled by holding all independent 

variables constant; then measure the bulb brightness 

Only a small percentage of students answered similarly to S3. Students would normally 
complete this activity during class in groups with great success, but surprisingly struggled as 
individuals. It is possible that stronger students were driving the group without any explicit 
discussion about the reasoning process, and that the high student-to-faculty ratio made it difficult 
for direct instructor intervention to ensure these discussions took place. By implementing pre¬ 
questioning, which is not necessarily built into some guided-inquiry curricula, we were able to 
identify student reasoning and it allowed us to explicitly address the various approaches as a class 
the next day. 
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If we categorize the five student responses into the approaches discussed in the previous 
sections, then we begin to see a pattern in the students’ approaches to hypothesis/model testing. 
S4 has devised a testing experiment that does not address the question, and describes the 
experiment through jargon that may or may not be appropriate, depending on the context of the 
students thinking, which we do not know. S5 recites previously learned jargon concerning the 
“scientific method,” such as independent and dependent variables, controlled experiments, etc. 
without explanation or definition within the immediate context. SI exhibits an appeal to the 
authority of a specific rule/law/model, person, or previous passive observation without providing 
a mechanism for new knowledge. 

S2’s response is more interesting because it has the ability to lead to new knowledge, and 
on the surface is a reasonable approach to addressing the research question. We see this line of 
student reasoning as an opportunity to have the students develop their own practice for evaluating 
evidence. Specifically, we developed a short guided-inquiry activity based around the proposed 
experiment provided by S2, which has been implemented in another class immediately after our 
discussion on the pre-question. By using an identical looking but nearly dead battery, or a bulb 
with a different rating, students are led to question the relative value of evidence, and begin the 
process of constantly thinking about their reasoning: developing good metacognitive practices. 
Our specific goal here is to guide students to a better understanding about the coordination of 
evidence and models, and towards an approach to science where they are constantly evaluating 
the relative value of evidence to support claims, as well as basic troubleshooting process. 

Authentic Research Experiences as a Capstone and for Independent Practice 

Small interventions like those discussed above lead into a larger capstone activity. At the end of 
the course, we have begun to incorporate an authentic research experience as described by 
Sikkema et al., where students investigate the motion of a spinning piece of PVC pipe (Sikkema, 
2010). Students are given a small segment of PVC pipe marked at opposite ends with different 
symbols. They then observe what happens when rotation is initiated by pushing one end 
downward. When spun in this way, one symbol on the tube will be visible while the other is not. 
Some students in our test group became very interested in the actual motion: did the tube merely 
process around an axis, or did the two sides “bobble” back and forth on the table. Video 
recordings seemed to suggest hobbling; however, this group devised a more definitive test. They 
painted one end with heavy marker ink and performed the experiment on a large sheet of white 
paper while the ink was still wet. They constructed the following “If ... and ... then ...” (IAT) 
statement (to be discussed in detail during the conference): 

“If the tube bobbles, and we ink one side, then a circle will be traced out on the page no 

mater what side is initially pressed.” 

The inked tube did trace a circle when pressed on the opposite end, but did not when 
pressed on the same end as the ink. They concluded that the hobbling model was falsified. This 
approach represents a change in epistemological resources: from passive observation to active 
experimentation in an attempt to falsify. Furthermore, it demonstrates the students’ reflecting on 
the nature of what constitutes proof. Specifically, this group initially assumed that the tube 
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hobbled. When presented with evidence that initially appeared to support this assumption (video 
recording), they were still able to evaluate the relative value of the evidence and its limitations 
and determine that a better experiment specifically designed to falsify the statement was 
necessary. This is exactly the type of approach to science that we want students to take away 
from the course. 

In the first instance with the light bulb task, we see students struggling in many ways: 
distinguishing the relative value of evidence, coordinating evidence with models, shifting 
epistemological resources, etc. After explicit instruction via context-rich activities on scientific 
reasoning, we begin to discern a shift from transitional to formal operational reasoning. 

Furthermore, by incorporating explicit instruction on reasoning patterns, it is possible to 
increase gains in scientific reasoning as measured by the LCTSR. Table II shows the average 
normalized gain on the LCTSR for populations receiving and not receiving explicit interventions 
like those discussed in this paper. An average normalized gain of 68% (n=14, s.d.=8%) was 
achieved on the LCTSR in comparison to 6% (n=62, s.d.=8%) for previous courses taught 
similarly with respect to content, though lacking explicit reasoning intervention. 



g 

N 

St. dev. 

non-explicit 

0.06 

62 

0.08 

explicit 

0.68 

14 

0.08 


Table II: Average normalized gain on the LCTSR for non-explicit 
and explicit reasoning intervention (Moore & Rubbo, 2012). 


Summary 

In summary, we have determined that the majority of non-STEM students can be classified as 
either concrete operational or transitional reasoners in Piaget’s theory of cognitive development, 
whereas in the science student population formal operational reasoners are far more prevalent. 
Through the incorporation of context-rich activities, authentic research experiences, and explicit 
interventions on reasoning patterns, we have been able to increase gains in student scientific 
reasoning abilities as well as transition students from transitional reasoners to more formal 
operational reasoners. 
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